Determining a need for a workspace graphical notation to increase user engagement

ABSTRACT

Approaches presented herein enable enhancing user engagement during a video conference. A user introduces (e.g., creates) a graphical notation during the video conference. If a correlation exists between the graphical notation and the video conference, then the graphical notation can be dynamically shared with any participants found to be disinterested or confused as a motivation to understand a concept or to act as a catalyst for additional graphical notations.

The present patent document is a continuation of U.S. patent applicationSer. No. 16/013,352, filed Jun. 20, 2018, entitled “DETERMINING A NEEDFOR A WORKSPACE GRAPHICAL NOTATION TO INCREASE USER ENGAGEMENT”, thedisclosure of which is incorporated herein by reference.

TECHNICAL FIELD

The subject matter of this invention relates generally to contentdelivery. More specifically, aspects of the present invention provide asolution for enhancing user engagement during a video conference.

BACKGROUND

Computing devices can be accessed at almost any time and any place andcan contain a tremendous amount of information relating to people,organizations, general interests, etc. In fact, many people choose toshare information about themselves with others, such as through a socialnetwork or through a journal. Such information can include thoughts,interests, photos, and anything else someone wants to share. Inaddition, organizations retain information relating to an organizationalhierarchy, job titles and descriptions, as well as information relatingto each employee, projects, competitors, and so forth.

In various organizations today, there is a tremendous amount of timespent preparing for meetings and attending those meetings, as well asother tasks related to meetings (e.g., action items). Sometimes meetingscan be productive and worthwhile for both the participant and theorganization. Meetings should allow participants the opportunity toachieve common goals while mitigating the amount of wasted time,regardless if the attendees are sitting around a table or around thecountry. Remote meetings have become an essential part of a workflow, oreven the only way of communication in various teams across the globe.Keeping meeting participants focused and engaged is vital to theproductivity of the session. Meetings should be more than merelyisolated events, but should be a piece of an overall collaborativeframework or organizational goal.

SUMMARY

In general, embodiments of the present invention provide for enhancinguser engagement during a video conference. A user introduces (e.g.,creates) a graphical notation during the video conference. If acorrelation exists between the graphical notation and the videoconference, then the graphical notation can be dynamically shared withany participants found to be disinterested or confused as a motivationto understand a concept or to act as a catalyst for additional graphicalnotations.

One aspect of the present invention includes a computer-implementedmethod for increasing user engagement in a video conference, comprising:estimating a user engagement status of a first participant; identifying,based on the user engagement status, a second participant who hasintroduced a graphical notation; determining whether a correlationexists between the graphical notation and the video conference;prompting, when a correlation exists between the graphical notation andvideo conference, the second participant to share the graphical notationwith the first participant; and dynamically displaying, in response toan affirmative response from the second participant, the graphicalnotation to the first participant.

Another aspect of the present invention includes a computer programproduct embodied in a computer readable medium that, when executed by acomputer device, performs a method for increasing user engagement in avideo conference, the method comprising: estimating a user engagementstatus of a first participant; identifying, based on the user engagementstatus, a second participant who has introduced a graphical notation;determining whether a correlation exists between the graphical notationand the video conference; prompting, when a correlation exists betweenthe graphical notation and video conference, the second participant toshare the graphical notation with the first participant; and dynamicallydisplaying, in response to an affirmative response from the secondparticipant, the graphical notation to the first participant.

Yet another aspect of the present invention includes a system forincreasing user engagement in a video conference, comprising: a memorymedium comprising instructions; a bus coupled to the memory medium; anda processor coupled to the bus that when executing the instructionscauses the system to perform a method, comprising: estimating a userengagement status of a first participant; identifying, based on the userengagement status, a second participant who has introduced a graphicalnotation; determining whether a correlation exists between the graphicalnotation and the video conference; prompting, when a correlation existsbetween the graphical notation and video conference, the secondparticipant to share the graphical notation with the first participant;and dynamically displaying, in response to an affirmative response fromthe second participant, the graphical notation to the first participant.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

These and other features of this invention will be more readilyunderstood from the following detailed description of the variousaspects of the invention taken in conjunction with the accompanyingdrawings in which:

FIG. 1 shows an architecture 10 in which the invention may beimplemented according to an illustrative embodiment of the presentinvention;

FIG. 2 depicts a cloud computing environment according to anillustrative embodiment of the present invention;

FIG. 3 depicts abstraction model layers according to an illustrativeembodiment of the present invention;

FIG. 4 shows a first schematic diagram 200 illustrating an exemplaryenvironment for implementation according to an illustrative embodimentof the present invention;

FIG. 5 shows an example display 300 of a user participating in a videoconference according to an illustrative embodiment of the presentinvention;

FIG. 6 shows an example display 400 including graphical notation 404according to an illustrative embodiment of the present invention;

FIG. 7 shows an example display 500 including pop-up box 502 accordingto an illustrative embodiment of the present invention; and

FIG. 8 shows an example process flowchart 600 for determining, based ona dynamic analysis of user engagement during a video conference, a needfor a workspace graphical notation to increase user engagement in thevideo conference according to an illustrative embodiment of the presentinvention.

The drawings are not necessarily to scale. The drawings are merelyrepresentations, not intended to portray specific parameters of theinvention. The drawings are intended to depict only typical embodimentsof the invention, and therefore should not be considered as limiting inscope. In the drawings, like numbering represents like elements.

DETAILED DESCRIPTION

Illustrative embodiments will now be described more fully herein withreference to the accompanying drawings, in which illustrativeembodiments are shown. It will be appreciated that this disclosure maybe embodied in many different forms and should not be construed aslimited to the illustrative embodiments set forth herein. Rather, theseembodiments are provided so that this disclosure will be thorough andcomplete and will fully convey the scope of this disclosure to thoseskilled in the art.

Furthermore, the terminology used herein is for the purpose ofdescribing particular embodiments only and is not intended to belimiting of this disclosure. As used herein, the singular forms “a”,“an”, and “the” are intended to include the plural forms as well, unlessthe context clearly indicates otherwise. Furthermore, the use of theterms “a”, “an”, etc., do not denote a limitation of quantity, butrather denote the presence of at least one of the referenced items.Furthermore, similar elements in different figures may be assignedsimilar element numbers. It will be further understood that the terms“comprises” and/or “comprising”, or “includes” and/or “including”, whenused in this specification, specify the presence of stated features,regions, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components, and/orgroups thereof.

Unless specifically stated otherwise, it may be appreciated that termssuch as “processing”, “detecting”, “determining”, “evaluating”,“receiving”, or the like, refer to the action and/or processes of acomputer or computing system, or similar electronic data center device,that manipulates and/or transforms data represented as physicalquantities (e.g., electronic) within the computing system's registersand/or memories into other data similarly represented as physicalquantities within the computing system's memories, registers or othersuch information storage, transmission, or viewing devices. Theembodiments are not limited in this context.

Graphical notations, colloquially referred to as “doodles”, can begenerally defined as simple drawings that can have concreterepresentational meaning or may just be composed of random and abstractlines. In the first context, doodling can have practical and powerfulapplications. Mathematicians and scientists can use graphical notationsto explain complex theories and equations. Business people can usegraphical notations to map business plans and strategies. People fromall walks of life are doodling to help them communicate and to givevisual representation and meaning to their ideas and to help others.

As stated above, embodiments of the present invention provide forenhancing user engagement during a video conference. A user introduces(e.g., creates) a graphical notation during the video conference. If acorrelation exists between the graphical notation and the videoconference, then the graphical notation can be dynamically shared withany participants found to be disinterested or confused as a motivationto understand a concept or to act as a catalyst for additional graphicalnotations.

Referring now to FIG. 1, a computerized implementation 10 of anembodiment for determining, based on a dynamic analysis of userengagement during a video conference, a need for a workspace graphicalnotation to increase user engagement in the video conference will beshown and described. Computerized implementation 10 is only one exampleof a suitable implementation and is not intended to suggest anylimitation as to the scope of use or functionality of embodiments of theinvention described herein. Regardless, computerized implementation 10is capable of being implemented and/or performing any of thefunctionality set forth hereinabove.

In computerized implementation 10, there is a computer system/server 12,which is operational with numerous other general purpose or specialpurpose computing system environments or configurations. Examples ofwell-known computing systems, environments, and/or configurations thatmay be suitable for use with computer system/server 12 include, but arenot limited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

This is intended to demonstrate, among other things, that the presentinvention could be implemented within a network environment (e.g., theInternet, a wide area network (WAN), a local area network (LAN), avirtual private network (VPN), etc.), a cloud computing environment, acellular network, or on a stand-alone computer system. Communicationthroughout the network can occur via any combination of various types ofcommunication links. For example, the communication links can compriseaddressable connections that may utilize any combination of wired and/orwireless transmission methods. Where communications occur via theInternet, connectivity could be provided by conventional TCP/IPsockets-based protocol, and an Internet service provider could be usedto establish connectivity to the Internet. Still yet, computersystem/server 12 is intended to demonstrate that some or all of thecomponents of implementation 10 could be deployed, managed, serviced,etc., by a service provider who offers to implement, deploy, and/orperform the functions of the present invention for others.

Computer system/server 12 is intended to represent any type of computersystem that may be implemented in deploying/realizing the teachingsrecited herein. Computer system/server 12 may be described in thegeneral context of computer system executable instructions, such asprogram modules, being executed by a computer system. Generally, programmodules may include routines, programs, objects, components, logic, datastructures, and so on, that perform particular tasks or implementparticular abstract data types. In this particular example, computersystem/server 12 represents an illustrative system for determining,based on a dynamic analysis of user engagement during a videoconference, a need for a workspace graphical notation to increase userengagement in the video conference. It should be understood that anyother computers implemented under the present invention may havedifferent components/software, but can perform similar functions.

Computer system/server 12 in computerized implementation 10 is shown inthe form of a general-purpose computing device. The components ofcomputer system/server 12 may include, but are not limited to, one ormore processors or processing units 16, a system memory 28, and a bus 18that couples various system components including system memory 28 toprocessor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Processing unit 16 refers, generally, to any apparatus that performslogic operations, computational tasks, control functions, etc. Aprocessor may include one or more subsystems, components, and/or otherprocessors. A processor will typically include various logic componentsthat operate using a clock signal to latch data, advance logic states,synchronize computations and logic operations, and/or provide othertiming functions. During operation, processing unit 16 collects androutes signals representing inputs and outputs between external devices14 and input devices (not shown). The signals can be transmitted over aLAN and/or a WAN (e.g., T1, T3, 56 kb, X.25), broadband connections(ISDN, Frame Relay, ATM), wireless links (802.11, Bluetooth, etc.), andso on. In some embodiments, the signals may be encrypted using, forexample, trusted key-pair encryption. Different systems may transmitinformation using different communication pathways, such as Ethernet orwireless networks, direct serial or parallel connections, USB,Firewire®, Bluetooth®, or other proprietary interfaces. (Firewire is aregistered trademark of Apple Computer, Inc. Bluetooth is a registeredtrademark of Bluetooth Special Interest Group (SIG)).

In general, processing unit 16 executes computer program code, such asprogram code for determining, based on a dynamic analysis of userengagement during a video conference, a need for a workspace graphicalnotation to increase user engagement in the video conference, which isstored in memory 28, storage system 34, and/or program/utility 40. Whileexecuting computer program code, processing unit 16 can read and/orwrite data to/from memory 28, storage system 34, and program/utility 40.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia, (e.g., VCRs, DVRs, RAID arrays, USB hard drives, optical diskrecorders, flash storage devices, and/or any other data processing andstorage elements for storing and/or processing data). By way of exampleonly, storage system 34 can be provided for reading from and writing toa non-removable, non-volatile magnetic media (not shown and typicallycalled a “hard drive”). Although not shown, a magnetic disk drive forreading from and writing to a removable, non-volatile magnetic disk(e.g., a “floppy disk”), and/or an optical disk drive for reading fromor writing to a removable, non-volatile optical disk such as a CD-ROM,DVD-ROM, or other optical media can be provided. In such instances, eachcan be connected to bus 18 by one or more data media interfaces. As willbe further depicted and described below, memory 28 may include at leastone program product having a set (e.g., at least one) of program modulesthat are configured to carry out the functions of embodiments of theinvention.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium including, but not limited to, wireless,wireline, optical fiber cable, radio-frequency (RF), etc., or anysuitable combination of the foregoing.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation. Memory28 may also have an operating system, one or more application programs,other program modules, and program data. Each of the operating system,one or more application programs, other program modules, and programdata or some combination thereof, may include an implementation of anetworking environment. Program modules 42 generally carry out thefunctions and/or methodologies of embodiments of the invention asdescribed herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a consumer to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via I/O interfaces22. Still yet, computer system/server 12 can communicate with one ormore networks such as a local area network (LAN), a general wide areanetwork (WAN), and/or a public network (e.g., the Internet) via networkadapter 20. As depicted, network adapter 20 communicates with the othercomponents of computer system/server 12 via bus 18. It should beunderstood that although not shown, other hardware and/or softwarecomponents could be used in conjunction with computer system/server 12.Examples include, but are not limited to: microcode, device drivers,redundant processing units, external disk drive arrays, RAID systems,tape drives, and data archival storage systems, etc.

Referring now to FIG. 2, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as private, community,public, or hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms, and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 2 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 3, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 2) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 3 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include mainframes. In oneexample, IBM® zSeries® systems and RISC (Reduced Instruction SetComputer) architecture based servers. In one example, IBM pSeries®systems, IBM System X® servers, IBM BladeCenter® systems, storagedevices, networks, and networking components. Examples of softwarecomponents include network application server software. In one example,IBM WebSphere® application server software and database software. In oneexample, IBM DB2® database software. (IBM, zSeries, pSeries, System x,BladeCenter, WebSphere, and DB2 are trademarks of International BusinessMachines Corporation registered in many jurisdictions worldwide.)

Virtualization layer 62 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers;virtual storage; virtual networks, including virtual private networks;virtual applications and operating systems; and virtual clients.

In one example, management layer 64 may provide the functions describedbelow. Resource provisioning provides dynamic procurement of computingresources and other resources that are utilized to perform tasks withinthe cloud computing environment. Metering and pricing provide costtracking as resources are utilized within the cloud computingenvironment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.Consumer portal provides access to the cloud computing environment forconsumers and system administrators. Service level management providescloud computing resource allocation and management such that requiredservice levels are met. Service Level Agreement (SLA) planning andfulfillment provides pre-arrangement for, and procurement of, cloudcomputing resources for which a future requirement is anticipated inaccordance with an SLA.

Workloads layer 66 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation; software development and lifecycle management; virtualclassroom education delivery; data analytics processing; and transactionprocessing. Further shown in workloads layer 66 is user engagementenhancement, which represents the functionality that is provided underthe embodiments of the present invention. As mentioned above, all of theforegoing examples described with respect to FIG. 3 are illustrativeonly, and the invention is not limited to these examples.

It is understood that all functions of the present invention asdescribed herein typically may be performed by the commandidentification functionality (of management layer 64, which can betangibly embodied as modules of program code 42 of program/utility 40(FIG. 1). However, this need not be the case. Rather, the functionalityrecited herein could be carried out/implemented and/or enabled by any ofthe layers 60-66 shown in FIG. 3.

It is reiterated that although this disclosure includes a detaileddescription on cloud computing, implementation of the teachings recitedherein are not limited to a cloud computing environment. Rather, theembodiments of the present invention are intended to be implemented withany type of networked computing environment now known or laterdeveloped.

Referring now to FIG. 4, a block diagram 200 describing thefunctionality discussed herein according to an embodiment of the presentinvention is shown. It is understood that the teachings recited hereinmay be practiced within any type of computing environment (e.g.,computer system 12). To this extent, the teachings recited herein may bepracticed within a stand-alone computer system or within a networkedcomputing environment (e.g., a client-server environment, peer-to-peerenvironment, distributed computing environment, cloud computingenvironment, and/or the like). If the teachings recited herein arepracticed within a networked computing environment, each physical serverneed not have a user engagement enhancement mechanism 50 (hereinafter“system 50”). Rather, system 50 could be loaded on a server orserver-capable device that communicates (e.g., wirelessly) with thephysical server for enhancing user engagement during a video conference.In a typical embodiment, however, system 50 or components of system 50may reside within a screen connected to an electronic device capable ofperforming the functions described herein.

Regardless, as depicted, system 50 can be implemented as program/utility40 on computer system 12 of FIG. 1 and can enable the functions recitedherein. It is further understood that system 50 can be incorporatedwithin or work in conjunction with any type of system that receives,processes, and/or executes commands with respect to IT resources in anetworked computing environment. Such other system(s) have not beenshown in FIG. 4 for brevity purposes. As shown, system 50 includes userengagement component 52, graphical notation recognition component 54,graphical notation correlation component 56, and graphical notationdissemination component 58. The functions/acts of each component isdescribed in detail below.

User engagement component 52, as executed by computer system/server 12,is configured to estimate a participant engagement status during a videoconference. A video conference is a live, visual connection between twoor more participants residing in separate locations for the purpose ofcommunication. At its simplest, video conferencing provides transmissionof static images and text between two locations. At its mostsophisticated, it provides transmission of full-motion video images andhigh-quality audio between multiple locations.

For each participant sharing video in a video conference, an associatedvideo channel can be analyzed using facial recognition technology todetermine an engagement status (e.g., actively participating,disinterested, confused, etc.) of the participant with regard to thevideo conference. Since existing facial recognition technology candetect facial features and movements, it can also ascertain emotions ormental states. If a participant's mouth curls upwards, for instance, itcan be inferred that the participant is happy since the facial movementindicates a smile or laughter. Facial recognition technology can be usedon a video sequence to track the fluctuations and strengths of emotionover time, and even capture “microexpressions,” or little flickers ofemotion that pass over a participant's face before she can controlherself or is even aware she has registered an emotion.

Alternatively or in addition, user engagement component 52 may beconfigured to analyze each video channel to detect physical indicia ofinterest of a video conference participant. In an embodiment,determining physical indicia of interest includes determining theposition of the eyes of a participant using gaze tracking techniques.For example, user engagement component 52 uses the position andorientation of each eye to determine where the participant is lookingrelative to a display used in the video conference. By determining wherethe participant is looking, user engagement component 52 can determinewhether the participant is focusing on the display. If the participantis determined to be focusing on the display, the user engagementcomponent 52 determines on what portion of the screen the participant isfocusing. User engagement component 52 then uses this information toestimate a level of engagement for the participant with regard to thevideo conference. For example, a participant's facial expression andmovement can indicate disinterest or confusion during an ongoing videoconference.

In another embodiment, the physical indicia of interest determined fromthe visual information includes a position of a participant's head. Byanalyzing the position of the participant's head, user engagementcomponent 52 is able to estimate where the participant is looking and,consequently, determine whether the user is looking at the display. Userengagement component 52 then estimates user engagement in the videoconference. In yet other embodiment, the determined physical indicia ofinterest can include a body lean of a participant. For example, aparticipant slouched over his desk can indicate disinterest in the videoconference. In any case, user engagement 52 may estimate, based on aparticipant's facial expressions and movements and/or physical indiciaof interest, an engagement status for a participant sharing video duringa video conference.

Graphical notation recognition component 54, as executed by computersystem/server 12, is configured to detect when a graphical notation(e.g., a graphical notation) has been introduced during a videoconference. In an example, a user may create a graphical notation inreal time during a video conference using an application installed onher device. The application may be provided strictly fordoodling/sketching or simply have doodling functionality along withother functionality. The graphical notation may be created using aninput device and/or one or more virtual drawing tools (e.g., a linetool, a circle tool, a freehand drawing tool, a text tool, etc.). Inanother example, a user may open an existing graphical notation using anapplication capable of presenting the graphical notation to the user ona display. The existing graphical notation can include one that has beenscanned by the user from a hard copy. The hard copy can include anoriginal document, paper copy, printout, and/or any record that can beread without the use of any device. Similarly, the existing graphicalnotation may include a photo of an existing graphical notation, eitherfrom a screenshot or hard copy graphical notation. In any case,graphical notation recognition component 54 can detect when a graphicalnotation has been introduced by a user during a video conference.

Graphical notation correlation component 56, as executed by computersystem/server 12, is configured to identify whether a correlation existsbetween a graphical notation (e.g., a graphical notation) and a videoconference. For example, a video conference including severalparticipants may be conducted to discuss altering an existing businessprocess. As the video conference begins, assume an initial flow diagramof the existing business process is shared to a device of eachparticipant as a basis for the discussion. Assume that a conferenceparticipant, Joan, begins doodling using a doodling application on herdevice. The graphical notation is detected and graphical notationcorrelation component 56 determines whether a correlation exists betweenJoan's graphical notation and the meeting. If no correlation exists, noaction is taken regarding the graphical notation. These steps aredescribed in greater detail below.

FIG. 5 shows an example display 300 of a user participating in a videoconference. As shown, display 300 shows the other participants in themeeting: Abe 302A, Carl 302B, Betty 302C, and Debbie 302D. Assume thatJoan (not pictured) is using display 300. Also shown is diagram 304which is shared among the participants. Diagram 304 shows an initialflow diagram of a business process. As stated, the discussion focuses onpotential changes that can be made to the business process. Assume theconversation takes several twists causing some confusion in Abe 302A asto what has been agreed upon. User engagement component 52 identifiesthe confusion with Abe 302A based on his facial expressions and/or bodylanguage. Further, graphical notation recognition component 54 detectsthat Joan is doodling directly on her instance of diagram 304 as thediscussion proceeds. Graphical notation correlation component 56 thendetermines whether a correlation exists between her graphical notationand the ongoing discussion.

Graphical notation correlation component 56 may identify whether acorrelation exists between a graphical notation (e.g., a graphicalnotation) and a video conference. If so, the graphical notation could beused as a motivation to understand a concept or to act as a catalyst foradditional graphical notations, In an embodiment, a clusteringtechnique, such as as K-means clustering, can be used to make such adetermination regarding correlation. Graphical notation correlationcomponent 56 can use any number of points (or coordinates) from anynumber of previous graphical notation and/or existing images as abaseline. Previous graphical notation and/or existing images for thispurpose may reside in a baseline database (not shown). Baseline databasecan use any type of database structure (e.g., relational, hierarchical,etc.) to store previous graphical notation (e.g., graphical notations),existing images, any associated text related to the graphicalnotations/images, and/or the like.

As additional lines/sketches are drawn, graphical notation correlationcomponent 56 can use the baseline to determine if the new graphicalnotation is aligned to the initial baseline. In an embodiment, cartesiancoordinates and their co-variances can be used to determine whether analignment exists between the new graphical notation and the initialbaseline. If there is an alignment, this can serve as evidence to infera correlation between the current graphical notation and previousgraphical notations and/or existing images. If there is a lack ofalignment, this can serve as evidence to infer a lack of correlation.

As stated, k-means clustering may be used to determine a correlationrelated to a new graphical notation. K-means clustering is a type ofunsupervised learning, which is used against unlabeled data (i.e., datawithout defined categories or groups). The goal of this algorithm is tofind groups in the data, with the number of groups represented by thevariable K. The algorithm works iteratively to assign each data point toone of K groups based on the features that are provided. Data points areclustered based on feature similarity. The results of the K-meansclustering algorithm include the centroids of the K clusters, which canbe used to label new data, and labels for the training data (i.e., eachdata point is assigned to a single cluster). Rather than defining groupsbefore looking at the data, clustering allows for finding and analyzingthe groups that have formed organically. Each centroid of a cluster is acollection of feature values which define the resulting groups.Examining the centroid feature weights can be used to qualitativelyinterpret what kind of group each cluster represents. In any case, aclustering technique can be in this context to determine whether eitheran existing graphical notation could be used as a motivation tounderstand a concept or to act as a catalyst for additional graphicalnotations.

FIG. 6 shows an example display 400 of a user (Joan, in this example)participating in a video conference. As shown, display 400 includesgraphical notation 404 made by Joan during the discussion. As previouslymentioned, assume user engagement component 52 has determined Abe 302Ais experiencing confusion based on his facial expressions and/or bodylanguage. Further, graphical notation recognition component 54 hasdetected Joan has created graphical notation 404 on her instance ofdiagram 304 as the discussion proceeds. Graphical notation correlationcomponent 56 then determines whether a correlation exists between Joan'sgraphical notation 404 and the video conference. To that end, Joan'sgraphical notation 404 is compared against a baseline to determinewhether Joan is altering the initial flow diagram in a meaningful way(e.g., adding process blocks, adding decision points, etc.) or sketchingsomething unrelated to the meeting, such as a cartoon character. In thisexample, the initial flow diagram (or subsections of the initial flowdiagram) can be used as the baseline. Alternatively, a common image type(i.e., a generic workflow) could be used as a baseline. Graphicalnotation 404 is assessed to infer deviation from the baseline. In anembodiment, an image cross-correlation function can be used forquantifying a difference between the graphical notation and othergraphical notations/images In another embodiment, any method, now knownor later developed, for quantifying a difference may be used.

Alternatively or in addition, any text in graphical notation 404 can becompared against video conference text (e.g., a meeting description,meeting notes, etc.) and/or metadata/text related to a baseline toassist in deriving a deviation. Any natural language processing (NLP)techniques, now known or later developed, may be employed to determinesimilarity and/or relatedness of the text. In any case, if the deriveddeviation is below a predefined threshold, a correlation between thegraphical notation and video conference is assumed. Otherwise, it isdetermined to be unrelated. If related, the graphical notation can beused as a motivation to understand a concept or to act as a catalyst foradditional graphical notations.

Graphical notation dissemination component 58, as executed by computersystem/server 12, is configured to, based on a user response, share agraphical notation with one or more other video conference participants.In an embodiment, upon determining that a graphical notation (e.g., agraphical notation) may be useful to other participants in a videoconference, graphical notation dissemination component 58 can query auser who has created the graphical notation as to whether the graphicalnotation may be shared with the others. FIG. 7 shows an example display500 for asking a user (e.g., Joan) who has created a graphical notationif she would like to share with the other participants. As shown,display 500 includes pop-up box 502 including the query and yes/noanswer options. If Joan answers affirmatively, then graphical notationdissemination component 58 may display graphical notation 404 to each ofthe other participants in the video conference.

In another embodiment, a user having been determined as confused may beasked if he would like to see a graphical notation that anotherparticipant has created to increase his understanding. Referring back tothe earlier example, user engagement component 52 determined Abe 302Awas confused. In this case, graphical notation dessimiation component 58may query Abe 302A (e.g., via pop-up box) as to whether he would like tosee graphical notation 404 to help his understanding of what is beingdiscussed. If he selects yes', graphical notation disseminationcomponent 52 can present graphical notation 404 on his display.

In yet another embodiment, as user having been determined asdisinterested or unfocused during the video conference may be asked ifshe would like to create a graphical notation (e.g., a graphicalnotation) to help her become more engaged. Assume user engagementcomponent 52 has identified Betty 302C as being disinterested during themeeting based on her facial expressions and body language. Graphicalnotation dissemination component 58 may query Betty 302C (e.g., viapop-up box) whether she would like to create a graphical notation tohelp her become more engaged. If she agrees, graphical notationdissemination component 58 can display an existing graphical notation orpart of an existing graphical notation on her display to get herstarted.

Referring now to FIG. 8, in conjunction with FIG. 4, an implementationof a process flowchart 600 for determining, based on a dynamic analysisof user engagement during a video conference, a need for a workspacegraphical notation (e.g., a graphical notation) to increase userengagement in the video conference is shown. At 602, user engagementcomponent 52 determines whether a participant is disinterested orconfused during a video conference. At 604, graphical notationrecognition component 54 detects when another participant has introduced(e.g., creating, uploading, etc.) a graphical notation during theconference. At 606, graphical notation correlation component 56determines whether the graphical notation is correlated to theconference. If so, at 608, graphical notation dissemination component 58prompts the participant who has introduced the graphical notationwhether he would like to share the graphical notation with the others tofacilitate a greater understanding and/or to increase engagement in theconference. If so, at 610, graphical notation dissemination component 58displays the graphical notation to the other participants of the videoconference.

Process flowchart 600 of FIG. 8 illustrates the architecture,functionality, and operation of possible implementations of systems,methods, and computer program products according to various embodimentsof the present invention. In this regard, each block in the flowchartmay represent a module, segment, or portion of code, which comprises oneor more executable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the blocks might occur out ofthe order depicted in the Figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently. It willalso be noted that each block of flowchart illustration can beimplemented by special purpose hardware-based systems that perform thespecified functions or acts, or combinations of special purpose hardwareand computer instructions.

Some of the functional components described in this specification havebeen labeled as systems or units in order to more particularly emphasizetheir implementation independence. For example, a system or unit may beimplemented as a hardware circuit comprising custom VLSI circuits orgate arrays, off-the-shelf semiconductors such as logic chips,transistors, or other discrete components. A system or unit may also beimplemented in programmable hardware devices such as field programmablegate arrays, programmable array logic, programmable logic devices, orthe like. A system or unit may also be implemented in software forexecution by various types of processors. A system or unit or componentof executable code may, for instance, comprise one or more physical orlogical blocks of computer instructions, which may, for instance, beorganized as an object, procedure, or function. Nevertheless, theexecutables of an identified system or unit need not be physicallylocated together, but may comprise disparate instructions stored indifferent locations which, when joined logically together, comprise thesystem or unit and achieve the stated purpose for the system or unit.

Further, a system or unit of executable code could be a singleinstruction, or many instructions, and may even be distributed overseveral different code segments, among different programs, and acrossseveral memory devices. Similarly, operational data may be identifiedand illustrated herein within modules, and may be embodied in anysuitable form and organized within any suitable type of data structure.The operational data may be collected as a single data set, or may bedistributed over different locations including over different storagedevices and disparate memory devices.

Furthermore, systems/units may also be implemented as a combination ofsoftware and one or more hardware devices. For instance, program/utility40 may be embodied in the combination of a software executable codestored on a memory medium (e.g., memory storage device). In a furtherexample, a system or unit may be the combination of a processor thatoperates on a set of operational data.

As noted above, some of the embodiments may be embodied in hardware. Thehardware may be referenced as a hardware element. In general, a hardwareelement may refer to any hardware structures arranged to perform certainoperations. In one embodiment, for example, the hardware elements mayinclude any analog or digital electrical or electronic elementsfabricated on a substrate. The fabrication may be performed usingsilicon-based integrated circuit (IC) techniques, such as complementarymetal oxide semiconductor (CMOS), bipolar, and bipolar CMOS (BiCMOS)techniques, for example. Examples of hardware elements may includeprocessors, microprocessors, circuits, circuit elements (e.g.,transistors, resistors, capacitors, inductors, and so forth), integratedcircuits, application specific integrated circuits (ASIC), programmablelogic devices (PLD), digital signal processors (DSP), field programmablegate array (FPGA), logic gates, registers, semiconductor devices, chips,microchips, chip sets, and so forth. However, the embodiments are notlimited in this context.

Any of the components provided herein can be deployed, managed,serviced, etc., by a service provider that offers to deploy or integratecomputing infrastructure with respect to a process for determining,based on a dynamic analysis of user engagement during a videoconference, a need for a workspace graphical notation to increase userengagement in the video conference. Thus, embodiments herein disclose aprocess for supporting computer infrastructure, comprising integrating,hosting, maintaining, and deploying computer-readable code into acomputing system (e.g., computer system/server 12), wherein the code incombination with the computing system is capable of performing thefunctions described herein.

In another embodiment, the invention provides a method that performs theprocess steps of the invention on a subscription, advertising, and/orfee basis. That is, a service provider, such as a Solution Integrator,can offer to create, maintain, support, etc., a process for determining,based on a dynamic analysis of user engagement during a videoconference, a need for a workspace graphical notation to increase userengagement in the video conference. In this case, the service providercan create, maintain, support, etc., a computer infrastructure thatperforms the process steps of the invention for one or more consumers.In return, the service provider can receive payment from the consumer(s)under a subscription and/or fee agreement, and/or the service providercan receive payment from the sale of advertising content to one or morethird parties.

Also noted above, some embodiments may be embodied in software. Thesoftware may be referenced as a software element. In general, a softwareelement may refer to any software structures arranged to perform certainoperations. In one embodiment, for example, the software elements mayinclude program instructions and/or data adapted for execution by ahardware element, such as a processor. Program instructions may includean organized list of commands comprising words, values, or symbolsarranged in a predetermined syntax that, when executed, may cause aprocessor to perform a corresponding set of operations.

The present invention may also be a computer program product. Thecomputer program product may include a computer readable storage medium(or media) having computer readable program instructions thereon forcausing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network (for example, the Internet, a local area network, awide area network and/or a wireless network). The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and routes the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises andocument of manufacture including instructions which implement aspectsof the function/act specified in the flowchart and/or block diagramblock or blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus, or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

It is apparent that there has been provided herein approaches fordetermining, based on a dynamic analysis of user engagement during avideo conference, a need for a workspace graphical notation to increaseuser engagement in the video conference. While the invention has beenparticularly shown and described in conjunction with exemplaryembodiments, it will be appreciated that variations and modificationswill occur to those skilled in the art. Therefore, it is to beunderstood that the appended claims are intended to cover all suchmodifications and changes that fall within the true spirit of theinvention.

What is claimed is:
 1. A computer program product embodied in a computerreadable medium that, when executed by a computer, performs a method forincreasing user engagement in a video conference, the method comprising:estimating a user engagement status of a first participant; identifying,based on the user engagement status, a second participant who hasintroduced a graphical notation; determining whether a correlationexists between the graphical notation and the video conference;prompting, when a correlation exists between the graphical notation andvideo conference, the second participant to share the graphical notationwith the first participant; prompting, when a correlation exists betweenthe graphical notation and the video conference, the first participantto view the graphical notation, wherein the correlation exists betweenthe graphical notation and the video conference when a deviation betweenthe graphical notation and a baseline is below a predefined threshold;and dynamically displaying, in response to an affirmative response fromthe second participant, the graphical notation to the first participant.2. The computer program product of claim 1, wherein introducing agraphical notation includes creating the graphical notation during thevideo conference.
 3. The computer program product of claim 1, whereinwhat the user engagement status includes is selected from a groupcomprising disinterest or confusion.
 4. The computer program product ofclaim 1, further comprising deriving the deviation, wherein the derivingincludes performing a clustering technique.
 5. The computer programproduct of claim 1, wherein the baseline is derived based on one or moreimages.
 6. A system for increasing user engagement in a videoconference, comprising: a memory medium comprising instructions; a buscoupled to the memory medium; and a processor coupled to the bus thatwhen executing the instructions causes the system to perform a method,comprising: estimating a user engagement status of a first participant;identifying, based on the user engagement status, a second participantwho has introduced a graphical notation; determining whether acorrelation exists between the graphical notation and the videoconference; prompting, when a correlation exists between the graphicalnotation and video conference, the second participant to share thegraphical notation with the first participant; prompting, when acorrelation exists between the graphical notation and the videoconference, the first participant to view the graphical notation,wherein the correlation exists between the graphical notation and thevideo conference when a deviation between the graphical notation and abaseline is below a predefined threshold; and dynamically displaying, inresponse to an affirmative response from the second participant, thegraphical notation to the first participant.
 7. The system of claim 6,wherein introducing a graphical notation includes creating the graphicalnotation during the video conference.
 8. The system of claim 6, whereinwhat the user engagement status includes is selected from a groupcomprising disinterest or confusion.
 9. The system of claim 6, furthercomprising deriving the deviation, wherein the deriving includesperforming a clustering technique.
 10. The system of claim 6, whereinthe baseline is derived based on one or more images.